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Abstract 


3 
This study employed a multiple, regression to predict examinees' differential 
* ; 
performance when tested by familiar and unfamiliar examiners. Subjects were 


preschool and school-age handicapped children, each of whom\had been tested on 
ty 


the Clinical Evaluation of Language Functions, once by a familiar and once by 
an unfamiliar tester, within a crossover design. Teachers' ratings, subjects’ 


self-reports, an anxiety scale, and a sociometric procedure accounted for 
. ‘La 
nearly 40% of the variance in differential test performance. Implications 


for valid assessment of handicapped children are discussed. : 
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Prediction of Suboptimal Test ‘Performance among 


Handicapped Children: An Exploratory Investigation 


There is donadderapie evidence (e.g., Fuchs, Featherstone, Garwick, & 
Fuchs, 1984; Fuchs, uclins Garwick, & Featherstone, 1983; Olswang & Carpenter, 
1978; Stoneman & Gibson, 1978) that he unfamiliarity of an examiner negatively 
attacks handicapped children's optimal absolute performance. Moreover, a 
recent ovesiigatl ah (Fuchs, Fuchs, Power, & Dailey, 1983) demonstrated that 
tester unfamiliarity depresses handicapped, but not nonhandicapped, children's 
test performance, thereby indicating an examiner's unfamiliarity constitutes a 
negatively biasing condition and threatens the validity of handicapped students' 
test performance. 

This evidence supports the view that examiners be familiar with handicap- 
ped children prior to testing. Whereas testers might view this prescription as 
asrieedhaaiiie sound, it also is likely many would ere as infeasible 
since examiners frequently operate within severe time constraints (ade AERA, . 
APA, & NCME, 1983, p- 14-2). However, if it an possible to predict which 
handicapped siuasens are likely to perform suboptimal ly with an unfamiliar 
examiner relative. to performance one a familiar examiner, then testers might 
establish pretest eonbact with snly a subgroup of these pupils. 

The purpose of this investigation was to begin to develop a basis for such 
a prediction. Previous pbadlettyalceines have been sporadic aindd inconclusive _ 
(cf. Feldman & Marites. eoits vache, ‘POR Garwick, & Featherstone, 1983; 
Fuchs, Featherstone, Garwick, & Fuchs, 1981). Also, they typically have in- 
cluded a Eeucher rating as the only prediction hues In contrast, the pres 
ent investigation sought to predict children's. differential test. performance on 


the basis of information (a) gathered from multiple informants (i-e., parents, 


peers, subjects, and teachers) and (b) generated by qualitatively different 


a 


iva 


Prediction 
2 


e 


procedures (i.e., parent ratings, peer nominations, self-reports, and teacher 


ratings). ‘ 6 4 oe 
" Method 
Subjects 
Subjects were 32 handicapped .children from a larger study (Fuchs et al., , 


1983), in which 64 subjects comprised four groups: 16 handicapped preschoolers 


(HP), 16 handicapped school-age children (HS), 16 nonhandicapped preschoolers, 
(NP), and 16 nomhandicapped school-age children (NS). The HPs and HSs were 
moderately to profoundly speech and/or language impaired, perforned within the ~, 


normal range on individually administered IQ tests, and were participating in 


language programs (preschool or elementary school levels) in the same public 


a educational collaborative. NPs and NSs were drawn from a large college- 
affiliated nursery school and public eleneritagy school, respectively. All 
subjects were Caucasian, English-speaking, and from predominantly middle-class , 
families ‘Theabad in 5 contiguous towns in Central Massachusetts. The mean CAs 
for handicapped and nonhandicapped subjects —s 77.44 (SD = 24,.91)-and 76.91 
(SD = 24.49) months, respectively. Average CAs for preschool and school-age 
subjects were 57.16 (SD = 7.20) and 97.19 (SD = 18.48) months , respectively. 
Whereas a two-way analysis. of variance (ANOVA; handicapped VS nonhandicapped 
and preschool vs. school-age) revealed a significant dif ference between the CAs 
of preschool and school-age subjects, F (1,60) = ius p< -001, there was no 
significant disparity in CA between handicapped ai nothandiqhpped dhitdven,? vee 
F(1,60) = .02, ns, and no significant interaction F(1,60) = .00, ns. Addition- | 


ally, identical numbers of male (N = 10) and female (N = 6) subjects consti- 


tuted the four subject groups. 
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As part of this larger study, each handicapped and nonhandicapped subject 
had been tested by 2 of 32 speech clinicians on the.Clinical Evaluation of 
Language Functions. Each.subject was tested twice, once by a familiar exaniter 
and once by an unfamiliar tester, within a crossover design. Also, the order in 
which examiners vested Fauitter and unfamiliar subjects and the time of day gabe 
jects were apeavaed were controlled. Findings of this larger study, indicated 
that, in comparison to nonhandicapped children, handicapped subjects performed 
significantly stronger when tested by examiners with whom they were fintier 
There was no significant interaction between familiarity and cA or among famil- 


° 


iarity, handicapping condition, and CA. 


Measures ? ; ‘ 


' The Clinical Evaluation of Language Functions (CELF; Semel-Mintz & Wiig, 


1982) is a esuprehensive language test comprised of two scales, processing 
(PS; i-e., auditory comprehension) and production (PC; i.e., verbal expression). 
Semel-Mintz and Wiig (1982) wepnuied Gnbemnal consistency of .82 and a test- 
retest reliability seaketetant of .96. Pilot administrations of the CELF were 
conducted vith Language-impaired preschoolers (CAs: 3-10’ to 4-8) to seats 
the possibility of a "Eloor” effect. Results indicated the CELF was 
appropriately difficult for the study's pe, Examiners ware trained to 
administer the CELF in one 3-hour training session, which wae concubed by a 
; v4 
certified speech clinician. | 
Iowa Social Competency Scales: Maternal Preschool and School-age Forms 
(Lowa; Pease, Clark, & Crase, 1982) assesses children's compliancé in social 
situations with parents, siblings, peers, and unfamiliar persons. Specific- 
ane the seeectaok form explores whether children are sia activating, 


' iepereannded va, or adaptive in unfamiliar situations aadloe with unfamiliar 
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persons; capable of sharing; . and cooperative. The school-age form exanines 
children's behavior with respect to task oriéntation, egocentricity, leader- 
ship, physical aiedey: affection toward parents, and apprehension. - The 
frequency with which these behaviors are displayed is rated by ‘mothers on a 
scale of, 1’ (infrequent) to 5 (always). For the Preschool Forn, Peape ‘et’ als 
(1982) report median Spearman-Brown reliability estimates for ‘total and ane 
variances to be -/8 and .75, respectively. Pease et al. obtained similar re-_ 
liability estimates for the School-age Form: -75 and .74 for total and unique 
-variances,- respectively.” , ‘ “. . ae 
detiatead Competence Scale for Children (PCSC; Harter, 1982) is a self- 
report measure developed for children ‘between 8 and 15 years. It explores 
respondents! self-esteem (PCSC-E) and perceived wedeeeenes in the anes 
(PCSC-C), social (PCSC-S), and physical (PCSC-P) domains. Harter (1982) 
reports Cronbach's alpha of 73, »76, «78, and .83 for each of these subscales, 
geduEebivalys The Pictorial Scale of Parngivied Competence and Acceptance for 


Young Children (Harter & Pike, 1981) comprises two additional forms: "P-K” for 


ages 4 and 5, and "1-2" for 6 and 7 year olds. These focus on areas of compe- 


tence very similar to those constituting the PCSC.. Whereas children respond to 
2 . % e ‘ 2 


written statements on the PCSC, the latter two forms employ a pictorial format. 
Harter and Pike (1981) report overall Cronbach's alpha of 88 and -87 fot the 


“P-K" and "1-2" ‘forms, respectivelys No distinction will be made below between 


Harter's parallel scales for younger and older children; all will be identified 
| - 


as "PCSC.” 


. 
' 


« Teacher Rating Scale of Children's Actual Competence (TRS; Harter, 1982) 


parallels the PCSC in substance. The present study employed the cognitive 


(TRS-C), social (TRS-S), and physical: (TRS-P) subscales.. On the TRS, teachers 


~ 
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rate children on a scale of 1 (least competent), to 4 (most competent).: In this 
investigation, the handicapped children's special education teachers completed 
the TRS. Harter (1982) reports internal soetaceday (KR20) coefficients of 
-96, .93, and .94 for the TRS-C, TRS-S, and TRS-P subscales, respectively. 


Revised Children's Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 


1978) comprises 28 anxiety-related questions, such as "It's hard for me to get 


to sleep at night,” “I feel sick to my stomach a lot," and “A lot of people 


don't like me." The questions are read to the children and are scored either 


a | js 
"Yes" or "No" by the examiner. An internal consistency (KR20) coefficient of 


.83 was reported (Reynolds & Richmond, 1978). 2 
puctompbets data were collected by means of a nominative anpEean) that was 
‘adapted from Hymel and Asher (1947). Preschool Sud iddarwarten subjects end 
their head eanped classmates selected three peers with whom they most liked | 
to play, or Positive Nomination (PNOM), and another three students with whom 
they least liked to play, or Negative Nomination (NNOM). School-age subjects 
and their handicapped classmates nominated three children from their class—- 
room for the following six categories: Like’ Cand don't like) to work with in 
class; like Gand don't like) to play with on the playground; and like (and 
don't like) to invite home after school. Preschool and school-age subjects’ 
-PNOM and NNOM scores were the total eee of nominations they received from 
their classmates. Asher, Singleton, Tinsley, and Hymel (1979) and Oden aad 
Asher (1977) provide various documentation for Ehe stability of these nom- 
inative procedures. ~ 7 . . 
Prdcedure 


Data collectors (DCs) were four female graduate and undergraduate stu- 


dents. They participated in five training sessions (14 hours), during which 


ec 
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they were trained to administer various instruments and procedures. One week 
ae s . 


after subjects were tested on the Clinical. Evaluation of Language Functions 


_(CELF), as’ part of the larger investigation, DCs adnbidwtered “the PCSC.. PNOM 
and NNOM procedures were conducted in small groups or individually, depending 
‘upon children's’ reading skills. At both the preschool and school-age special 
education sites, all of the subjects’ classmates participated in the nomina- 

tive procedures. The ‘TRS was distributed to the teacher with whom each aie 

ject spent the most time. The Iowa was completed by aihiowtet mothers or. 

; gataenal: gasvdtnuds The RCMAS was administered to subjects by their familiar 
suacuai clinician at the end of the administration of the CELF in the earlier 


phase of the investigation. . ; | a ce 


Results . o% ‘ 
The dependent variable of interest, the dttterende between children's 
performance in the familiar and unfamiliar test conditions, was converted to a 
regressed-adjusted bogeapdiaves variable to avoid problems inherent in working 
with difference scores (Cohen &-Cohen, 1975). In this process, scores in the 

unfamiliar sondt tien: the covariate set (A), were regressed,on scores in the 
familiar: condition, the dependent variable (Y), resulting in a new set of 
familiar condition scores. This new estimate then wad subtbacted eiek the 
original familiar condition score, creating a regressed-adjusted familiarity 
score (RAFS), which was entered iabeate analyses presented below. This 
process ee is referred to as “residualization" of Y~by A, “adjusting” Y 
for A, and Y “with A held conebane statistically.” 

Means and standard deviations are displayed in Table 1 for RAFS and the 
following variables: (a) PCSC-C, PCSC-P, PCSC-S, and PCSC-E subscales of the 


Perceived Competence Scale for Children and the Pictorial Scale of Perceived 


‘ 


ae 
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Competency and Acceptance for Young Children ("P-K" and "1-2");. (b) TRS-C, 
TRS-P, and TRS-S subscales af the Teacher Rating Scale of Children's Actual 
Competence; (c) sit (NNOM) and positive (PNOM) noMLNaL LOBES (d) Revised | 
Children's Manifest dnoeteey Scale (RCMAS); and (e) the Iowa Social a Sac 
Scales (Iowa) for praaehoul and Sahoeisade children. 


Insert Table 1 about here 


Correlational analyses were conducted ais the relations among ys 
RAFS and the 11 psychosocial variables. These Pearson saitdeia: 
: corelations- are shown in Table 2. Five variables correlated with RAFS at 
a value of at least +. ee (32) = 505 p = -002 for TRS-C; r(32) = .29, p = 
-055 for TRS-P; r(32) : = .26, p = .073 for NNOM; r(32) = -.33, p = .034 for 
CMAS; and r(32) = -.20, p = «137 for Iowa. . 


a ee ae a ee ee ee ee ae ee ee ee oe a me me 


Of these five variables, four were entered into a forward stepwise 


multiple regression to ‘determine what portion of the total variance in 

RAFS was accounted for by these predictors. TRS-P, the fifth variable, was 
excluded because of its high intercorrelations with three of the other 
variables entered into the multiple regression (with TRS-C, r(32) = .38, 

“a = .016; with NNOM, r(32) = -.25, p = shee otdh Iowa, r(32) = ae, 

j -021). Results of the multiple regression are shown in Table 3. TRS-C . 
accounted for 25% of the total variance, . followed by CMAS (6%), Iowa (5%), 


and NNOM (3%). The effects due to these four ‘variables were significant 


? Prediction 
; \ 8 
_ (see Table 3). Together, the four predictors accounted for 39% of the total 
variance in RAFS. 
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Insert Table 3 about here 


Discussion : ss 


Findings indicate moderate relations between children's differantial 
performance across familiar and unfamiliar examiner conditions and the follow- 
ing variables: the Cognitive subscale (TRS-C) of the Teacher Rating Scale of 
Children's Actual Competence, Negative Nomination (NNOM), Revised Children's 
Manifest Anxiety Scale (RCMAS), and the Iowa Social’ Competency Scales (Iowa). 

This suggests gavewel: dabinetive chacamnentstics of speech-and/or 
language-handicapped children whose differential performance in tuner of 
familiar examiners reflects suboptimal performance with unfamiliar examiners. 
First, they seem to be comparatively competent pupils, as indicated by the 
strong positive correlation between TRS-C and the regressed-adjusted fam- 
iliarity score (RAFS). Second, they appear to be relatively free of chronic 
anxiety, as evidenced by the inverse relation between RCMAS and RAFS. Last, 
they seem to have poor social skills, as ‘suggested by the high number of 
negative nominations they received from classmates and low ratings they were 
assigned by their mothers. Moreover, the negative correlation between RAFS 
and subjects’ cceieee of themselyes on the social’ subscale of the Perceived 
Competency Scale for Children suggests they are aware of their poor. 
social skills. Taken together, these results suggest a portrait of an exam- 
inee who, aieheak not skilled in social relations with peers and adults; has 
the capacity to recognize the familiar examiner as accepting and supportive 


( 
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. and, perhaps because of relatively low anxiety, can respond adaptively in the 
“familiar condition. 

The important role of cognitive competeyse tn any portrait is correkore> 
ted by Mischel (1973), who describes certain individuals' cross-situational 
variability as a highly pebined “discriminative facility.” Its apparent 
importance also is consonant with results from an investigation conducted by 
Harter (1967), who employed retarded, normal, and bright children and found — 
the normal and bright children learned more rapidly in a social condition, 
whereas the retarded children learned more quickly in a non-social condition. 

The four measures involved in this portrait, TRS-C, NNOM, RCMAS, and 
Iowa, we cuiieua for nearly -40% of the variance in differential partoraance 
with familiar and unfamiliar examiners. ' This dasuit compares piosiahis with 
previous investigations (e.gs, Fuchs, Fuchs, Garwick, & Featherstone, 1981) 
and should encourage continued work on the development of an instrument that 
would predict, among select groups, children whose test performance may be 
improved through familiarization with an examiner. 

Nevertheless, atatierteat and conceptual issues require words of cau-. 
tion. When multiple regression is used with a relatively anid wauie: the 
regression coefficients tend to be unstable from one sample to another 
(Kerlinger & Pedhazur, 1973). While the present use of multiple regression 
seems justified within the context of this exploratory edys replication with 
a larger sample is necessary. 

More theoretically, this study identified suboptimal performers by their 
eeeolieer performance with familiar testers. This is a limited conceptualiza~ 
tion of suboptimal performance because it precludes recognition of exintnens 


who may perform suboptimally in both familiar and unfamiliar examiner condi- - 


12 
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a tions. Further research on explicating and predicting suboptimal test per- 
%, x formance might employ alternate and more comprehensive definitions of the 


construct. . Ss : > 5 « 
| Second, the eeieteaetan of this investigation implies that speech-and/or .~ 
language-impaired children performed more strongly with familiar testers bias 
cause of factors sig cei? aaa themselves. However, recent evidence 
indicates examiners’ detliniike and behaviors sometimes affect children's test 


~ 


| performance (cf. Sattler, 1974). This bears on the nature of the above por- 
trait of examinees performing eee with familiar testers. Rather 
than view differential performance’as an expression of examinees' capacity to 
respond appropriately to subtle changes in situational seneinieuteus aidtex 
ia" | "ential performance with familiar examiners may relate to testers! selective, 
_ facilitating behavior. ny | . u 
in nae ees fantiiae with tha mova spunitinete senaveRne 
” handttapeed child may provide more encouragement (eeg., Thomas, Hertzig, 
‘Dryman, & Fernandez, 1971), permit greater amounts of response time (e.g-, 
Aili dg Zern, & Fuchs, 1983), or use less stringent scoring eptteria elses. 
a . Fuchs & Fuchs, in, press; Sattler & Winget, 1970). Future savciyh whaday 
~ address the challenging task of exploring simultaneously both examinee 


characteristics and ee behaviors to wnderauand better how to optimize 


handicapped chiideent s t@st PORE CE MANEGs: 3 
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Table 1- 
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Means and Standard Deviations on Measures 


i) 
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Measure Mean "Standard Deviation 
Brae % 
RAFS . 6363 ee! ae 
-PCSC-C 20631. 2.9 a 3613" 
PCSC-P_ 1963 eT 
- pesc-s : 19.00 426 
PCSC-E . 19.47 fe ge Bee 
TRS-C 16.34 ee ee ‘ . 
TRS-P 15.06 Ke 4.28 
TRS-S , e. 14.50 a 4-30 
NNOM © . 1506 . - 10675 ' 
PNOM ‘ 17.09. « , 97 , 
CMAS | 15625 6492. 
Lowa _ S536 . 0.47 
» F 
17 
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Table 2 
Intercorrélations between Test Measures 
Measure 1 2 2 eee 5 6 7 8 9 10 ll 12 
RAFS 
PCSC-C Ol 
PCSC-P 10 043 
PCSC-S -.14 20 oA ° + 
PCSC-E 09 044 «e50 647 4 
TRS-C 250 6B5, +20 215 | -.07 
ae | | 
TRS-P . . 29 ; -.05 19 “e ll _ 7208 38 | 
TRS-S 03. 027 « 33 02 0606 
- NNOM 26 03 -15 -.08° 21 413 -.25 -.12 
PNOM -.04-. -.18 09 -.10 -21 17 04 14 -.06 
CMAS -.33 -.03 -00 -ell -.10 -.18 «09 19 -ell 19 
lowa -.20 -~.09— -.16 00 03 -+02 -.36 -14 00 08 -.20 
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Table 3 : 


Regression Analysis of Predictors on Regressed-Adjusted Familiarity Score 


: * 


Source Multiple R R2 cumulative 2 Change B B F(1,30) p<, 
ee a a ee a ee ee a re re 
TRS-C 50 025 125 2-48 © 33 9.80 O01 
CMAS 255 031 -06 =~ P65 4 52 6.35 2025 
Iowa +60 36 .05 “14.84  -.20 5.32 05 
NNOM -63 * .39 03 0:72 > «22 4.37 05 


